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ABSTRACT 


Intelligent tutoring systems (ITSs) have been successful at 
improving students' performance across a variety of domains. To 
help achieve this widespread success, researchers have identified 
important behavioral and performance measures that can be used 
to guide instruction and feedback. Most systems, however, do not 
present these measures to the teachers who employ the systems in 
classrooms. The current paper discusses visualizations that will be 
displayed to teachers using the writing strategy tutor, Writing Pal. 
We present visualizations for both classroom and student level 
data and offer descriptions of each. 
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1. INTRODUCTION 


Over the past several decades, intelligent tutoring systems (ITSs) 
have been successfully developed for and implemented across a 
variety of domains [1]. These computer-based systems are often 
designed to record every interaction, behavior, and performance 
marker a student achieves while using the system. Research in 
educational data mining has used these system logs to identify 
what data are most predictive of overall performance and learning 
[2], while research in the learning sciences has used system logs 
to tailor instruction to individual students [3]. The synthesis of 
this work yields more adaptive, effective systems. 


The analysis of log data has helped develop complex 
computational algorithms that improve adaptability within ITSs 
by modeling the learner [4]. Learner models can be difficult to 
understand without experience in modeling and educational 
research, and as a result, researchers have developed visualization 
tools to render components of these models more accessible [5]. 
Such tools are important because of the potential disadvantages 
that may emerge when the teachers who use ITSs have little 
understanding of their underpinnings. For instance, teachers may 
be less likely to use a system if they do not understand a system's 
feedback or what drives the feedback [6]. Moreover, if a system 
does not convey appropriate and timely information about 
students, the instructor may be unable to intervene [7]. 


Visualizations provide one means of aiding teachers in 
deciphering the complexity of ITSs and making data-driven 
classroom decisions [e.g., 8]. Our team is working toward 
providing visualizations of student progress within the Writing 


Pal (W-Pal), a writing strategy ITS designed for high school 
students. Writing Pal provides strategy instruction via lesson 
videos, game-based strategy practice, and essay practice with 
automated, formative feedback [9]. In this paper, we describe 
visualizations we have developed and implemented as well as 
those we are currently prototyping. 


2. VISUALIZING DATA 


Our initial goal is to provide the most relevant and understandable 
data to teachers through intuitive visualizations. The following 
sections describe visualizations that we are developing for W- 
Pal’s teacher interface, where teachers view students’ progress. 


2.1 Classroom Level Visualizations 

In a recent classroom implementation of W-Pal, five ninth grade 
classes with the same teacher used the system for approximately 
four months. We analyzed data from 90 consenting students. For 
the study, W-Pal’s teacher interface included a spreadsheet in 
which teachers could track students’ progress through the system 
activities (see Figure 1). However, during the study, this page did 
not provide a visual summary of the progress across students. 
Broadcasting the average number of activities attempted in a 
classroom of students who have generally stalled in their 
progression might prompt teachers to request that students not 
linger on particular topics or switch their focus. Future iterations 
of W-Pal will provide easily discernible bars that indicate the 
overall progress of classes. In Figure 1, the darker blue bars in the 
first four columns represent the percentage of activities attempted 
for those modules (a black rectangle highlights this feature). 
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Figure 1. Visualization of a classroom’s progress in W-Pal’s 
teacher interface; dark blue bars represent progress. 


An important strength of W-Pal is the automated feedback it 
provides on students’ essays. The teacher interface allows teachers 


to view each student’s submitted essays along with the feedback 
and score received. Currently, however, teachers do not have 
access to a summary of all students’ performance. For example, if 
the majority of students are struggling to properly structure their 
writing in W-Pal, teachers would remain unaware until they 
carefully perused students’ feedback messages. To provide 
teachers with a quickly consumable summary of the feedback that 
students are receiving, we are developing a visualization that 
displays the percentage of feedback triggered across all essays in a 
W-Pal class (see Figure 2). Using this information, teachers might 
adjust their own classroom instruction or assign students to 
interact with appropriate W-Pal lessons. 
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Figure 2. Visualization of the type of essay feedback students 
in a classroom have received. 


2.2 Student Level Visualizations 

Our recent classroom study also revealed that the percentage of 
time that students selected different activities related to their 
persistence in the system. For example, there was a positive 
correlation between the percentage of game activities that students 
selected and the number of days they used the system [7(90) = .49, 
p < .001]. Thus, the percentage of activities attempted (i.e., 
videos, games, and essay practice) could be indicative of how 
likely students are to persist in the system. Teachers will be 
presented with this information via pie charts, which are useful for 
visualizing proportions of a whole [10] (see Figure 3). 


F CLASS INFO. 


Activity breakdown for Student X 


Exp 
Compare to: 

Your students’ average °°" 
W-Pal activities ™ 


Essays. 
8% 


Games. 
14% 


Figure 3. Visualization of the activity breakdown for an 
individual student. 


Similar to the activity breakdown available for each student, 
teachers will be able click students' names in the essay window to 
see breakdowns of essay feedback (see Figure 2 for a similar 
example). If a student is struggling with writing assignments in 
class, this visualization will give teachers a quick view of how W- 
Pal has assessed areas of weakness. 


3. CONCLUSION 


In this paper, we argue for the importance of using visualizations 
to communicate data from ITSs to the teachers. Specifically, we 
describe classroom and student level visualizations that we are 
developing for the writing strategy tutor, W-Pal. When equipped 


with these visualizations, teachers may be more likely to use a 
system appropriately and to intervene when a student is not 
performing optimally. Future empirical work must test these 
visualizations, through techniques ranging from surveys to eye 
tracking [8], to determine their effectiveness in conveying 
information to teachers. As the understanding of how teachers use 
such visualizations grows, systems can provide teachers with 
intelligent tutors that better support classroom instruction. 
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